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(54) Method and system for comicstrip representation of multimedia presentations 



(57) The present invention provides a method and 
process for an Internet WWW user to rapidly display on 
her browser, a high quality comicstrip type summary 
representation of a much larger multimedia work, which 



work would normally require significant time and net- 
work bandwidth to transmit to the user, wherein the com- 
icstrip summary includes an audio portion of the work 
related to the scenes selected for the comicstrip display 
and optionally a text summary of the story plot. 
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Description 

The present invention relates to a method and sys- 
tem for comicstrip representation of multimedia presen- 
tations. 

More particularly the present invention relates to the 
field of information retrieval. More specifically, the 
present invention relates toa client/server model for pro- 
viding a client on a network with a comicstrip type sum- 
mary of a much larger multimedia work which would or- 
dinarily require significant time and bandwidth to trans- 
mit to the client. 

Computer-to-computer communication networks 
are becoming larger and larger and message traffic on 
these networks is increasing exponentially. Moreover, 
technology companies are emphasizing a new channel 
- the Internet- for the distribution of goods, services and 
intellectual property. It is desirable therefore that mes- 
sage transmission on these networks be made as effi- 
cient as possible. 

The development of computerized information re- 
sources, such as the Internet, and various on-line serv- 
ices, such as Compuserve, America On-Line, Prodigy, 
and other services, has led to a proliferation of electron- 
ically-available information. In fact, this electronic infor- 
mation is increasingly displacing more conventional 
means of information transmission, such as newspa- 
pers, magazines, and even television and the movies. 
The primary issue in ail of these resources is a combi- 
nation of rapid transmission of the desired information 
so as not to waste the user's time waiting for a transmis- 
sion to complete, and how to select only the desired in- 
formation for transmission so as not to transmit unnec- 
essary data which takes unnecessary transmission time 
(and related cost) and uses an unnecessary amount of 
the available network bandwidth. It is desirable that 
there exist a process for allowing a user on the Internet 
to rapidly display a representation or summary of a mul- 
timedia work without having to download the entire 
work. 

For example, in the case of multimedia works en- 
compassing video and audio, the digitized video portion 
of these works requires a great deal of computer mem- 
ory and a corresponding large amount of transmission 
system bandwidth. One frame of a standard video film 
requires approximately 900 kilobytes of computer mem- 
ory locations for storage, and an 8 minute video film, for 
example, requires 12 gigabytes of storage. Of course 
data compression schemes of various kinds are used 
for reducing the amount of storage required, for exam- 
ple, one common method would reduce the 8 minute 
video to 1 92 megabytes. Similarly, one frame of a com- 
pressed video would take approximately" 2 minutes to 
transmit across the United States, and the entire 8 
minute video would take approximately 27 hours using 
norma! modem speeds. Accordingly, when a user is try- 
ing to access information about available multimedia 
works on a remote server using the client/server func- 
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tionality provided by the World Wide Web ("WWW") and 
a well known browser, she is typically unwilling to sit and 
wait for more than a minute or two for the work to be 
downloaded to her system for viewing, especially if she 
s is not sure that she is interested in the work in the first 
place. 

In the past, attempts have been made to compress 
the video portions of such multimedia works in order to 
decrease the transmission bandwidth required as well 

10 . as the transmission time. Unfortunately these compres- 
sion schemes as yet are only able to produce playback 
images of such poor quality as to be unrepresentative 
of the quality of the original video, and to reduce trans- 
mission time by a factor of two so that a work that would 

'5 take 30 minutes to transmit will still take 15 minutes, 
which is still too long for a typical WWW user to wait. 
Thus for example if the particular system were seeking 
to facilitate the purchase or rental of a multimedia work, 
the compressed version's poor quality and still lengthy 

20 download time would seriously impede the purchase/ 
rental decision. One such attempt to use compression 
for such displays was to ... (discuss closest Prior art in 
more detail) 

Another earlier case in which people have attempt- 
's ed to offer multimedia works for sale/rental was merely 
to download a television listing, including available mov- 
ies for viewing to a subscriber's television receiver (see 
U.S. Patent No. 4,706,1 21 ). Such listings are easily dis- 
played on a browser on the WWW as well, however list- 
30 ings of events are not as effective as viewing a comics 
trip type summary of the work especially if it has accom- 
panying audio. 

The present invention provides a method and proc- 
ess for an Internet WWW user to display on her browser, 
35 a high quality comicstrip type representation of a multi- 
media work which includes the audio portion of the work 
related to the scenes selected for the comics trip display 
as well as some limited text for describing plot summary, 
etc. 

40 

SUMMARY OF THE INVENTION 

The present invention overcomes the disadvantag- 
es of the above described systems by providing an eco- 

45 nomical, high performance, adaptable system and 
method for creating a comicstrip type summary of a larg- 
er multimedia work, with accompanying audio related to 
the summary strip of images, which can minimize user 
wait time and related transmission time and cost'when 

so attempting to view the summary on a World Wide Web 
("WWW") browser. The present invention is a computer 
controlled process and system for creating a digitized 
comicstrip representing a multimedia work, wherein a 
file of selected audio can be attached to the comicstrip. 

55 and wherein both the comicstrip image file and the re- 
lated audio file are optimized to reduce transmission 
time on the WWW without sacrificing image or audio 
quality. 
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In one aspect of the present invention a computer 
system is disclosed for use in providing a client on a 
network with a comicstrip summary of a much larger 
multimedia work, which multimedia work would require 
a significant time and network bandwidth to transmit to s 
the client if it were transmitted in its entirety, the compu- 
ter system including a server capable of receiving and 
storing a digitized version of a multimedia work ; an im- 
age selection mechanism configured to select a group 
of images from the multimedia work, and a comicstrip w 
processor which can form a comicstrip file from the 
group of images selected and which can be transmitted 
over the network to the client in a commercially accept- 
able amount of time. There is additionally disclosed an 
aj jdio selection mechanism with _which-to a sea- is 

ment of audio from the multimedia work to create a s ub- 
se t of the work's aud io file and to attach the subset file 
to the cornlcsfrip"fne" 

In another aspect of the invention, a computer im- 
plemented comics trip creation mechanism is disclosed, 20 
for use in providing a client on a network with a comic- 
strip summary of a much larger multimedia work, which 
multimedia work would require a significant time and 
network bandwidth to transmit to the client if it were 
transmitted in its entirety, having an image selection 25 
mechanism, a comics trip processor and an audio se- 
lection mechanism. 

In still another aspect of the invention, a method is 
disclosed for constructing a comicstrip of images from 
a multimedia work by providing devices to use in digitiz- 30 
ing the work, selecting a set of images from the work to 
represent the work in summary form and in creating a 
comicstrip out of the selected images for rapid transmis- 
sion to a user. 

In still another aspect of the invention, a method is 35 
disclosed of providing a client on a network with a com- 
icstrip summary of a much larger multimedia work 
wherein the comicstrip summary has an audio file relat- 
ed to the comics trip summary and wherein the comic- 
strip summary and related audio file can be transmitted 40 
to the client in a fraction of the time and using a fraction 
of the bandwidth required by the larger multimedia work 
by, selecting a set of images from the work to represent 
the work in summary form, limiting the use of colors in 
the colormap representing the color in the pixels of the 
images selected and creating a comics trip out of the 
selected images for rapid transmission to a user 

The present invention will now be further described, 
by way of example, with reference to the accompanying 
drawings, in which: - 50 

Figure 1 illustrates a client/server system in which 
an embodiment of the present invention may be im- 
plemented. 

Figure 2 illustrates a more detailed view of the cli- 
ent/server shown in Figure 1. 

Figure 3 illustrates the structure of a workstation in 
which the client or server may be operative. 



Figure 4 illustrates the concept of a movie/video 
comicstrip. 

Figure 5 illustrates a typical movie/video/multime- 
dia digitization process. 

Figure 6 is a flow chart of the process of creating a 
comicstrip file with the related audio file. 
Figure 7 illustrates a view of the processes opera- 
tive in a server supporting a multimedia work order 
or display comicstrip process. 
Figure 8 is a flow chart of the user procedure for 
accessing the related browser, selecting a comic- 
strip summary, playing the related audio and possi- 
bly ordering the work. 

Figure 9 illustrates an exemplary comicstrip type 
display with accompanying plot summary. 
Figure 10 illustrates a typical HTML script used to 
attach the related audio file to a comics trip summa- 
ry and to attach the related plot summary to the 
comics trip display. 

Figure 11 illustrates an exemplary browser display 
window showing an exemplary descriptive ad for a 
film with hypertext links to other pages. 
Figure 12 illustrates an exemplary browser display 
window for a page of additional scenes for the- film 
of Fig. 11 which was made to appear by clicking on 
a hypertext pointer on the previous window. "*This 
window also contains hypertext pointers to other 
pages, including one to "a "comic book" witrv still 
photos and the sound track" 

Figure 13 illustrates an exemplary browser display 
window was made to appear by clicking on the hy- 
pertext pointer on the previous window which point- 
er to the "comic book". This window contains an ex- 
emplary "comics trip", plot text and a hypertext 
pointer to the "soundtrack" associated with the com- 
icstrip. 

Figure 14 illustrates an exemplary browser display 
window showing again the window containing the 
comicstrip of Fig. 13 but also displaying an audi- 
otool window which accompanies the playing of the 
soundtrack related to the comicstrip. 

NOTATIONS AND NOMENCLATURE 

The detailed descriptions which follow are present- 
ed largely in terms of algorithms and symbolic represen- 
tations of operations on data bits within a computer, 
memory. These algorithmic descriptions and represen- 
tations are the means used by those skilled in the data 
processing arts to most effectively convey the sub- 
stance of their work to others skilled in the art. 

An algorithm is here, and generally, conceived to be 
a self-consistent sequence of steps leading to a desired 
result. These steps are those requiring physical manip- 
ulations of physical quantities. Usually, though not nec- 
essarily, these quantities take the form of electrical or 
magnetic signals capable of being stored, transferred, 
combined, compared, and otherwise manipulated. It 
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proves convenient at times, principally for reasons of 
common usage, to refer to these signals as bits, values, 
elements, symbols, characters, terms, numbers, or the 
like. It should be bourne in mind, however, that all of 
these and similar terms are to be associated with the 
appropriate physical quantities and are merely conven- 
ient labels applied to these quantities. 

Further, the manipulations performed are often re- 
ferred to in terms, such as adding or comparing, which 
are commonly associated with mental operations per- 
formed by a human operator. No such capability of a 
human operator is necessary, or desirable in most cas- 
es, in any of the operations described herein which form 
part of the present invention: the operations are ma- 
chine operations. Useful machines for performing the 
operations of the present invention include general pur- 
pose digital computers or similar devices. In all cases 
there should be bourne in mind the distinction between 
the method operations in operating a computer and the 
method of computation itself. The present invention re- 
lates to method steps for operating a computer in 
processing electrical or other (e.g., mechanical, chemi- 
cal) physical signals to generate other desired physical 
signals. 

The present invention also relates to apparatus for 
performing these operations. This apparatus may be 
specially constructed for the required purposes or it may 
comprise a general purpose computer as selectively ac- 
tivated or reconfigured by a computer program stored 
in the computer. The algorithms presented herein are 
not inherently related to a particular computer or other 
apparatus. In particular, various general purpose ma- 
chines may be used with programs written in accord- 
ance with the teachings herein, or it may prove more 
convenient to construct more specialized apparatus to 
perform the required method steps. The required struc- 
ture for a variety of these machines will appear from the 
description given. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

The present invention provides a method and proc- 
ess for an Internet WWW user to rapidly display on her 
browser, a high quality comicstrip type summary repre- 
sentation of a much larger multimedia work which work 
would normally require significant time and network 
bandwidth to transmit to the user, wherein the comicstrip 
summary includes an audio portion of the work related 
to the scenes selected for the comicstrip display and op- 
tionally a text summary of the story plot. 

Although the following will be described with refer- 
ence to certain particular embodiments. incju^]ng_data 
structures, flow of steps, hardware configurations, etc, 
it will be apparent to one skilled in the art that implemen- 
tations of the present invention can be practiced without 
these specific details. 

Implementations of the present invention use a cli- 
ent/server architecture, as illustrated in Figure 1 , where- 



in user requests 1 1 0 for information are sent by a client 
application program 100 to a server 150 (typically, a re- 
mote computer system accessible over the Internet or 
other communication medium). The server 150, as will 

$ be described in more detail below, performs scanning 
and searching of multimedia information sources (e.g. 
libraries of movies, vtdoes, musicals, etc.), based upon 
these user requests, presents the filtered electronic in- 
formation ( i.e. comicstrip summaries with related audio) 

J0 as server responses 1 60 to the client process. The client 
process may be active in a first computer system, and 
the server process may be active in a second computer 
system, and communicate with one another over a com- 
munication medium, thus providing distributed function- 
's ality and allowing multiple clients to take advantage ol 
the information gathering capabilities of the server. 

A more detailed view of the client and server are 
shown in Figure 2. Although the client and server are 
processes which are operative within two computer sys- 

20 terns, these processes being generated from a high lev- 
el programming language (e.g. C or C++), compiled and 
executed in a computer system (e.g. a workstation), it 
can be appreciated by one skilled in the art that they 
may be implemented in a variety of hardware devices, 

25 either programmed or dedicated. 

Client 100 and server 150 communicate using the 
functionally provided by the World-Wide Web (WWW). 
Clients and servers of the WWW communicate over a 
communication medium 250 using a standard known as 

30 the Hypertext Transfer Protocol (HTTP). In some em- 
bodiments, the client and server may be coupled via Se- 
rial Line Internet Protocol (SLIP) or TCP/IP connections 
for high- capacity communication. Active within the cli- 
ent is a first process, known as a "browser" 200, which 

35 establishes the connection with server 150, and 
presents information to the user. Any number of com- 
mercially or publicly-available browsers may be used, 
in various implementations, however in this implemen- 
tation, browser 200 is the Netscape Navigator brand 

<to browser, available from Netscape Communications 
Corporation, Mountain View, California. Other browsers 
such as the Mosaic browser (version 2.0 or greater) 
available from the National Center for Supercomputing 
Applications (NCSA) in Urbana-Champaign, Illinois, 

4 $ Netcruiser, or others, which are available and provide 
the functionality specified under the HTTP protocol and 
the Netscape brand browser or above may be used. 

The server 150 executes the corresponding server 
software which presents information to the client in the 

50 form of HTTP responses. The HTTP responses corre- 
spond with the web "pages' represented using Hyper- 
text Markup Language (HTML), or other data which is 
generated by the server, as will now be discussed. 
A shortcoming of HTTP is that it is a stateless pro- 

55 tocol. The Web "page" from which user transactions 
may be performed in the server under control of the cli- 
ent is not recalled from client-session to client-session. 
As a result, functionality to effect transmission efficien- 
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cies requires an underlying mechanism which is unmet 
by current versions of HTTP. Under the Netscape Nav- 
igator, in addition to HTML functionality 2 1 0 provided by 
the server (display and retrieval of certain textual and 
other data based upon Hypertext views and selection of 
itom(s)), a Common Gateway Interface (CGI) 220 is pro- 
vided which allows the client program to direct the server 
to commence execution of a specified program con- 
tained within the server. Using this interface, and HTTP, 
the server may notify the client of the results of that ex- 
ecution upon completion. The server's application pro- 
gram, the multimedia work comicstrip summary gener- 
ator, maintains a file of the selected comicstrip frames 
and related audio, and thus, provides state functionality 
from session to session to an otherwise stateless pro- 
tocol. 

In alternate embodiments, it may be desirable to 
permit a user to specify some order of viewing or some 
limit in the amount of transmission time that the user is 
willing to permit for such a download action. In such a 
case, in order to control the parameters of the execution 
of this server-resident process, the client may direct the 
filling out- of certain "forms" from his browser. This is 
also provided by the "fill-in forms" functionality 230 avail- 
able under the Netscape Navigator or Mosaic browsers, 
which allows the user via his client application program 
100, to specify a "profile" in which the server will cause 
an application program to function (e.g. the types of sto- 
ries/articles which are of interest to the user, the trans- 
mission times permitted, etc.). The details of a user pro- 
file will be not be discussed herein howeverthose skilled 
in the art will readily understand how to use the "forms" 
capability to create such functionality. 

A computer system, such as a workstation, person- 
al computer or other processing apparatus in which the 
client 100 or server 150 may be operative is illustrated 
in Figure 3. A workstation in which one implementation 
of the present invention may be practiced includes sys- 
tem 300. System 300 comprises a bus or other commu- 
nication means 301 for communicating information, and 
a processing means 302 coupled with bus 301 for 
processing information. System 300 further comprises 
a random access memory (RAM) or other volatile stor- 
age device 304. (referred to as main memory), coupled 
to bus 301 for storing information and instructions to be 
executed by processor 302. Main memory 304 also may 
be used for storing temporary variables or other inter- 
mediate information during execution of instructions by 
processor 302. System 300 also comprises a read only 
memory (ROM) and/or other static storage device 306 
coupled to bus 301 for storing static information and in- 
structions for processor 302, and a data storage device 
307 such as a magnetic disk or optical disk and its cor- 
responding disk drive. Data storage device 307 is cou- 
pled to bus 301 for storing information and instructions. 
This may be used for storage of the digital image and 
related audio databases to be described here using 
commercially available software products. 



a 

System 300 may further be coupled to a display de- 
vice 321, such as a cathode ray tube (CRT) or liquid 
crystal display (LCD) coupled to bus 301 for displaying 
information to a computer user. Displays on other type 

s display devices will not limit the scope of the present 
invention. Such a display 321 may further be coupled to 
bus 301 via a frame buffer 310, which contains informa- 
tion such as a single or multiple frames or images for 
display upon display device 321 . An alphanumeric input 

10 device 322. including alphanumeric and other keys, may 
also be coupled to bus 301 for communicating informa- 
tion and command selections to processor 302. An ad- 
ditional user input device is cursor control 323, such as 
a mouse, a trackball, stylus, or cursor direction keys, 

is coupled to bus 301 for communicating direction infor- 
mation and command selections to processor 302, and 
for controlling cursor movement on display 321 . Audito- 
ry output may be played on a loudspeaker 326 connect- 
ed to system 300. 

20 Note, also, that any or all of the components of sys- 
tem 300 and associated hardware may be used in var- 
ious embodiments, however, it can be appreciated that 
any configuration of the system may be used for various 
purposes according to the particular implementation. 

25 in one embodiment, system 300 is one of the Sun 
Microsystems® brand family of workstations such as the 
SPARCstation brand workstation manufactured by Sun 
Microsystems® of Mountain View, California. Processor 
302 may be one of the SPARC brand microprocessors 

30 manufactured by Sun Microsystems®, Inc. of Mountain 
View, California. (Sun Microsystems is a registered 
trademark of Sun Microsystems, Inc.) 

Note that the various embodiments discussed here- 
in will refer specifically to a series of routines which are 

^5 generated in a high-level programming language (e.g., 
the C or C++ programming language) and compiled, 
linked, and then run as object code in system 300 during 
run-time, for example by the SPARCompiler available 
from SunS oft of Mountain View. California (SPARC and 

40 SPARCstation are trademarks of SPARC International, 
Inc. and are licensed exclusively to Sun Microsystems). 
These further are used in conjunction with the browser 
and server software available from Netscape, as de- 
scribed above, including the specification of the appear- 

45 ance of displays in HTML. It can be appreciated by one 
skilled in the art, however, that the following methods 
and apparatus may be implemented in special purpose 
hardware devices, such as discrete logic devices, large 
scale integrated circuits (LSI's), application-specific in- 

50 tegrated circuits (ASIC's), or other specialized hard- 
ware. The description here has equal application to ap- 
paratus having similar function. For example, in a com- 
puter system without a loudspeaker 326. the audio 
might be played through a regular telephone if the server 

55 150 is connected to the public telephone network and 
the user has entered her telephone number on a form 
230. 
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What Is the Invention 

Referring now to Fig. 4, the concept of a movie or 
video "Corniest rip summary" is described 400. Concep- 
tually, a comicstrip 430 is a "window" or "poster" con- 
taining a limited number of still images which, generally 
in sequence, represent a summary of a larger movie or 
video work. For example, the comicstrip 430 depicted 
contains still images 432 through 442 which correspond 
to specific frames of the film 420. That is frame 422 for 
example, corresponds to still image 432 on the comic- 
strip 430. frame 424 corresponds to still image 434 and 
so on. In the preferred embodiment, the comicstrip 430 
corresponds to a WWW browser window and the still 
images 432 through 442 would correspond to still imag- 
es of such a size to reasonably fit on the browser window 
while being large enough to show reasonably sharp de- 
tail of the image. Fig. 9 provides a sample of such a 
comicstrip created by the preferred embodiment. Re- 
turning to Fig. 4 for a moment, it is apparent in the case 
depicted that there is no sound associated with the com- 
ics trip as shown. In the preferred embodiment, as illus- 
trated in Fig. 5 the physical film 420 from Fig. 4 is digi- 
tized 510 by processor 520 and stored in mass storage 
530 by a program contained in local memory 540. In al- 
ternative embodiments, the film 420 may be read by 
processor 520 without first storing the entire digitized 
film in mass storage 530 but rather storing only selected 
frames of the film in either mass storage 530 or local 
memory 540. In the preferred embodiment the audio 
from the audio track of the film 420 is also captured by 
processor 520 and stored as a separate file which can 
be linked to the video image file. 

Turning now to Fig. 6 this selection process 600 is 
described in more detail. To begin, the film is loaded to 
the film digitizer 604. Commercially available equipment 
for reading the video or movie film and creating a digital 
file of the images, such as an Apple Macintosh® Quadra 
840 AV for example, can be used. (Note: Apple and 
Macintosh are registered trademarks of Apple Compu- 
ter, Inc.). In the preferred embodiment the built in digi- 
tizer of an Apple Macintosh Quadra 840 AV is used to 
create the digital image file. Typically the digitized im- 
ages are stored in mass storage 606. The digital images 
ol the video or movie are now displayed using a com- 
mercially available viewer program 608 and the frames 
selected for the comicstrip 610. In the preferred embod- 
iment a VtdeoFusion 1.5.1 viewer from Video Fusion, 
Inc., Maumee, Ohio is used. Such viewers usually have 
a slow-motion function and a "frame-step" function 
which facilitates selecting the best frame image for the 
comicstrip. Most such commercial viewers also have a 
"save image" function which facilitates saving the se- 
lected image to a separate file 612. In the preferred em- 
bodiment the frame images are further processed to re- 
duce the number of colors used in the images to 128 
613. In the preferred embodiment, Photoshop© 2.5.1 
from Adobe® Systems, Inc. is used. (Adobe is a regis- 
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lered trademark and Photoshop is a trademark of Adobe 
Systems, Inc.). This is done by mapping the color values 
of the pixets in the selected image to an equivalent color- 
map containing only 128 colors. This process does not 
5 diminish the clarity of the images unnecessarily and yet 
further reduces the number of bytes of data necessary 
to represent an image, thus assisting in the speed-up of 
any subsequent transmission of these images. The orig- 
inal film audio, which was also captured during the dig- 
itizing process 604 is now replayed in order to select 
audio segments appropriate for the images selected for 
the comicstrip 614,616. The audio file is compressed to 
6 bit quality audio further reducing the number of bytes 
necessary to represent the audio and yet not reducing 
the audio quality beyond acceptable limits. The com- 
pressed audio segments selected are stored to a sepa- 
rate audio file and the address of this audio file connect- 
ed to the comics trip 618. In the preferred embodiment, 
this connection is done using the "html" script shown in 
Fig. 10. This "html" file may also be used to retain and 
connect a short text description of the plot of the story 
displayed in the comicstrip or a scene setting, or what- 
ever description desired for this comicstrip. This is de- 
scribed in detail below. 

How to Use the Invention 

Having created a comicstrip with related audio file, 
how a user would access this mechanism using her 
browser is now described. Fig. 7 illustrates the interac- 
tion between the server 1 50 and video/film order or com- 
icstrip processor 710 program, which is operative under 
control of the CGI in the server 220. Depending upon 
user input, requests are sent to the active application in 
the server, in this case the video/film order or comicstrip 
processor 710 program, which causes the application 
to perform certain functions. If the request is a command 
to order a copy of the movie/film/multimedia work, the 
request is passed by the video/film order or comics trip 
processor 710 program to the order processor 720. If 
the request is a command to view the comicstrip sum- 
mary for a multimedia work, the request is passed by 
the video/film order or comicstrip processor 710 pro- 
gram to the comics trip processor mechanism 730 for 
selection of the designated work and transmission to the 
requester. This process is described in more detail from 
the viewpoint of the user with reference to Fig. 8. The 
user begins the process by bringing up her browser 804 
(see Fig. 11 for a typical view of the Netscape browser 
which is used in the preferred embodiment). The "dis- 
play video page" (which may be called by various 
names, such as shown in the example below) is brought 
up by the browser 806. The user has the option of mak- 
ing a selection (this is described relative to an example 
below in Figs. 11-14) lor ordering a copy of the desired 
multimedia work or requesting a comicstrip summary 
display of the work 808. If the "order" selection is made 
810 the system passes the order to the process video 
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order process 812 and the task exits 814 displaying a 
reply "order process completed" on the browser. If the 
comicstrip display request was selected (see 962 in Fig. 
12) 816 the system finds the selected comicstrip and 
related video files on the server according to the hyper- 
text command related to the selected indicator on the 
browser page (962 in Fig. 12) and downloads these files 
to the user's machine 818. The browser displays the 
comicstrip 820, as for example shown 920 in Fig. 9 and 
970 in Figs. 13 & 14. Typically a plot summary 930 in 
Fig. 9 is included with the comicstrip file for display on 
the browser with the comicstrip. Continuing in Fig. 8 the 
user may now download and play the audio file 822 
which accompanies the comics trip display file. This au- 
dio play activation is performed by clicking on the indi- 
cator ( 91 0 in Fig. 9 or 972 in Fig. 1 3) on the comicstrip 
page. Typically the user can replay the audio 826 by 
merely clicking again on the ■soundtrack" indicator (910 
in Fig. 9) or using the rewind and replay buttons on the 
audio tool (982 in Fig.14). Normally the user now has 
the option of ordering 832 the multimedia work whose 
comics trip she just viewed or selecting another comics 
trip to view 836, 840, or exiting the browser 838. 

Fig. 10 displays the "html" script used in the pre- 
ferred embodiment to attach the audio file and the plot 
summary to the comicstrip display window. Those 
skilled in the art will recognize that various other scripts 
and configurations could be used to accomplish these 
purposes. 

An Example 

An example of how the present invention would be 
used is now described with reference to Figures 11-14. 
Fig. 1 1 depicts a typical window using the WWW brows- 
er from Netscape. The window shown in Fig. 1 1 is a win- 
dow that would be in the "Technology & Research" sec- 
tion 940 of the "Home Page" of Sun Microsystems, Inc. 
which is found on the WWW at URL=http;//www.sun. 
com. This window shown in Fig. 11 is not yet in the Sun 
External Home Page but will be in the future. !n this ex- 
emplary page (Fig. 11) an ad for the film "Starfire" 942 
is shown. A picture (visual image) 944 of one of the 
scenes is shown with text 945 describing the film, and 
some text identifying certain other scenes from the film 
946. The scenes identified typically have hypertext 
pointers associated with each, as for example the 
"Boardroom" 948 scene indicator. Clicking the computer 
display cursor on this hypertext pointer for the Board- 
room scene 948 causes a new window to appear on the 
browser similar to Fig. 12. Fig. 12 is like the previous 
window in that the title 950, visual image 952 and other 
text 954 and 956 are displayed. In this case the text 
"Film clips" 956 lists other clips that the user can display 
such as "Quick Time movie (24 MB)" 958, "MPEG movie 
(3.8 MB)" 960 and "A "comic book' with still photos and 
the sound track (0.8 MB)" 962. It is this latter item which 
is the invention disclosed in this application. Clicking the 



computer display cursor on this hypertext pointer 963 
for the ""comic book" with still photos and the sound 
track (0.8 MB) H 962 causes a new window to appear on 
the browser similar to Fig. 1 3. In Fig. 13a sample com- 
5 icstrip of the type disclosed for the present invention 970 
is displayed which was produced from the film "Starfire". 
A hypertext pointer 972 indicates where the user should 
click to get the related audio file and the additional text 
which might go with the comicstrip 970 is displayed on 
10 the bottom of the page (not visible in this figure) 974. 
Clicking the computer display cursor on this hypertext 
pointer 972 for the "soundtrack" 972 causes a new win- 
dow to appear on the browser similar to Fig. 14. Actually 
the window shown in Fig. 13 containing the comics trip 
J 5 (970 in Fig. 1 3) remains on the computer display screen 
and a small window 976 appears which displays the 
controls of the audiotool which the user may have on 
her computer system. In this example, the audiotool 
contained in the Sun Microsystems Solaris operating 
20 environment is depicted 976. At the same time as the 
display appears the audiotool begins to play the audio 
from the audio file that was related to the comics trip. In 
this audiotool display is shown data indicating the audio 
is "voice format" 978, of length of time 1:04.7 minutes 
25 980. Additionally controls to adjust the volume of the au- 
dio 984 and to "stop play", "reverse or conceptually re- 
wind the audio" and "fast forward" 982 are shown. In this 
example there are no hypertext links shown with which 
the user could order a copy of the film, however in the 
30 preferred embodiment it is envisioned that such a choice 
would be made available to the user. 

While the present invention has been described in 
terms of a comicstrip summary of a larger multimedia 
work, the problem of minimizing the user wait time for a 
35 response on the WWW network is one which is present 
when any lengthy work or document is requested for 
downloading. Accordingly the process disclosed herein 
may be adapted to various other similar situations. For 
example, allowing users to judge a film or television 
JO show before ordering it through a pay-per-view service, 
helping judges in competitions like the Academy Awards 
to quickly get an overview of a large number of audio- 
visual works before choosing which ones to nominate 
or consider more closely for the award, allowing expa- 
ts triates to keep up with the evening news or other televi- 
sion shows (rom their home country, allowing activists 
to bring their case to a world audience where transmit- 
ting the full video of the case would take up so much 
. bandwidth as to make its transmission impossible, al- 
so lowing grandparents and other remotely located family 
members to get an impression of a birthday party or sim- 
ilar events that have been captured on home video. 
Those skilled in the art will recognize other examples of 
how such a comicstrip summary of a larger multimedia 
55 work would be beneficially used, and accordingly this 
list of exemplary uses is in no way to be considered ex- 
haustive or otherwise limiting. 

in addition, while the present invention has been de- 
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scribed with reference to Figures 1 through 1 1 , it will be 
appreciated that the figures are (or illustration only, and 
do not limit the spirit and scope of the invention. By ne- 
cessity, the specification describes, for example, screen 
images having certain attributes and features, however, 
it will be appreciated that the invention is not limited by 
the specific examples provided herein. 



Claims 



5. The computer system of claim 2 further comprising 
a request processor in said server system, said re- 
quest processor configured to transmit to a client 
said comicstrip and related audio file. 

5 

6. The computer system of claim 5 wherein said client 
comprises a display processor configured to make 
requests to said server system and to display said 
comicstrip to a user after said comics trip has been 

10 transmitted to said client by said server system. 



1. A computer system having a central processing unit 7. 
(CPU) and random access memory (RAM) coupled 

to said CPU, for use in providing a client on a net- 
work with a comicstrip summary ofa much larger is 
multimedia work, which multimedia work would re- 
quire significant time and network bandwidth to 8. 
transmit to the client, said computer system com- 
prising: 

20 

a server system configured to receive and store 
a digitized version of a multimedia work; 9. 
an image selection mechanism coupled to said 
server system, said image selection mecha- 
nism configured to permit selection of a plurality 25 
of images from said multimedia work; and 
a comicstrip processor coupled to said server 
system, said comicstrip' processor configured 
to create a comicstrip file for said multimedia 
work using said plurality of images from said 30 
multimedia work, whereby said comics trip file 
can be transmitted over the network to said cli- 
ent in a substantially reduced amount of time 
compared with said multimedia work. 

35 

2. The computer system of claim 1 further comprising 
an audio selection mechanism coupled to said serv- 
er system, said audio selection mechanism config- 
ured to permit a selection of audio from said multi- 
media work independent of said plurality of images -to 
selected, and to create a file of said selected audio 
and to relate said file of said selected audio to said 
comicstrip file. 

3. The computer system of claim 2 wherein said file of 4 $ 
said selected audio is compressed to 8 bit audio for- 
mat to further reduce amount of data and related 10. 
transmission time for transmitting said file of said 
selected audio to said client. 

50 

4. The computer system or claim 1 wherein said plu- 
rality of images from said multimedia work in said 11. 
comicstrip file are modified by reducing number of 

colors used to represent color of a pixel in said im- 
ages to 1 23 colors thereby further reducing amount 55 
of data and related transmission time for transmit- 
ting said comicstrip file of said selected images to 
said client. 



The computer system of claim 6 v/herein said client 
further comprises a selection mechanism config- 
ured to permit said user to play said related audio 
file while displaying said comicstrip. 

The computer system of claim 7 v/herein said client 
comprises an HTTP browser and said server sys- 
tem comprises an HTTP server application pro- 
gram. 

A computer implemented comics trip creation sys- 
tem for providing a client on a network with a com- 
icstrip summary of a much larger multimedia work, 
which multimedia work would require significant 
time and network bandwidth to be transmitted to 
said client, the comicstrip creation system compris- 
ing: 

an image selection mechanism configured to 
permit selection of a plurality of images from a 
digitized version of a multimedia work; 
a comicstrip processor coupled to said image 
selection mechanism, said comics trip proces- 
sor configured to create a comicstrip file for said 
multimedia work using said plurality of images 
from said multimedia work: and 
an audio selection mechanism coupled to said 
comicstrip processor, said audio selection 
mechanism configured to permit a selection of 
audio from said multimedia work independent 
of said plurality of images selected, and to cre- 
ate a file of said selected audio and to relate 
said file of said selected audio to said comic- 
strip file. 

The computer implemented comicstrip creation 
system of claim 9 further comprising a request proc- 
essor configured to transmit to a client said comic- 
strip file and related audio file. 

The computer implemented comicstrip creation 
system of claim 9 wherein said plurality of images 
from said multimedia work are modified by reducing 
number of colors used for pixels in said images to 
1 23 colors, thereby reducing number of bytes of da- 
(h needed to represent images in said comicstrip 
hie 
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12. The computer implemented comicstrip creation 
system of claim 9 wherein said selection of audio 
from said multimedia work is compressed and 
stored in 8 bit quality form, thereby reducing number 
of bytes of data needed to represent audio in said 
audio file. 

13. A computer implemented method for constructing a 
comicstrip of images from a multimedia work, said 
method comprising the steps of: 

providing a first device to use in storing a digi- 
tized version of a multimedia work in a memory 
coupled to said computer; 
providing a second device to use in selecting a 
plurality of images from said multimedia work; 
providing a third device to use in creating a 
comicstrip file for said multimedia work which 
uses said plurality of images from said multime- 
dia work; and 

providing a fourth device to use in storing said 
comicstrip file in said memory. 

14. The method of claim 1 3 further comprising steps of 
providing a mechanism to use in selecting audio 
from said multimedia work independent of said plu- 
rality of images selected, and providing a mecha- 
nism to use in creating a file of said selected audio 
and relating said file of said selected audio to said ' 
comicstrip. 

15. The method of claim 14 further comprising a step 
of providing a mechanism to be used in transmitting 
to a client said comicstrip and related audio file. 

16. The method of claim 15 further comprising a step 
of providing a device to be used in displaying said 
comicstrip to a user after said comicstrip has been 
transmitted to said client. 

17. The method of claim 16 further comprising a step 
of providing an audio playback tool to use in playing 
said related audio file while displaying said comic- 
strip. 

18. The method of claim 1 7 wherein said client compris- 
es an HTTP browser. 

1 9. The method of claim 1 3 wherein said plurality of im- 
ages from said multimedia work are modified by re- 
ducing a number of colors used to represent color 
in pixels in said images to 1 28 colors. 

20. The method of claim 13 wherein said selection of 
audio from said multimedia work is compressed and 
stored in 3 bit quality form. 

21. A method of providing a client on a computer net- 
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work with a comicstrip summary of a much larger 
multimedia work wherein the comicstrip summary 
. has an audio filed related to the comicstrip summary 
and wherein the comics trip summary and related 
audio file can be transmitted to the client in a fraction 
of the time and using a fraction of the bandwidth 
required by the larger multimedia work, said method 
comprising the steps of: 

selecting a limited number of images from the 
multimedia work by means of a computer 
mechanism; 

storing said limited number of images from the 
multimedia work in a computer memory; 
selecting a limited amount of audio from the 
multimedia work by use of a computer mecha- 
nism; 

relating the audio file' to the comicstrip file; and 
transmitting to the client over the network the 
comicstrip file and related audio file 

whereby the transmission of the comicstrip 
file and related audio file takes a fraction of the time 
and uses a fraction of the bandwidth that would be 
used by the larger multimedia work. ^ 

22. The method of claim 21 comprising the additional 
step of limiting the number of colors used to repre- 
sent color in pixels of the selected limited number 
of images from the multimedia work to a level less 
than 129 colors thereby reducing the bytes of data 
required to store said limited number of images from 
the multimedia work. 

23. The method of claim 21 comprising the additional 
steps of limiting the bit representation of the audio 
to a quality level less than 9 bits, thereby limiting the 
bytes of data required to store said limited amount 
of audio, and storing said limited amount of audio 
which has been reduced to 8 bit quality coding in an 
audio file in computer memory. 

24. The method of claim 21 wherein the audio fiie is re- 
lated to the comicstrip file using an HTML script. 

25. The method of claim 21 wherein a selected amount 
of text is related to the comicstrip file. 

26. The method of claim 25 wherein the selected 
amount of text is related to the comicstrip file using 
an HTML script. 

27. A browser window display to use in a computer dis- 
play device comprising: 

a comicstrip summary of a larger multimedia 
work comprising a limited number of images 
taken from the larger multimedia work wherein 
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Ihe limited number cf images have been limited 
to color representation of 128 or less colors; 
a hypertext pointer to an audio clip containing 
a selected portion of audio from said larger mul- 
timedia work wherein the selected portion of s 
audio is coded in 8 bit audio and is accessible 
by clicking on said hypertext pointer; and 
a text portion to use in describing a plot sum- 
mary of said story represented by said comic- 
strip summary or to use with other descriptive io 
text data 

wherein said comics trip summary, said audio 
clip and said text portion are downloaded to 
said browser window display from a remote 
computer device in a fraction of time it would 1$ 
take to download said multimedia work to the 
computer device containing the browser win- 
dow.. 
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Boardroom Meeting 

Click here for the soundtrack. 
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Julia is giving a multimedia presentation to the Board of Directors (where some of Ihe members 
are attending remotely). She is proposing that the company start manufacturing a car designed 
by her division. Her rival, Mike springs a surprise during the meeting and claims that Julie's car 
will not sell well since a previous attempt at a similar car failied. Julie uses her laptop computer to 
search a remote information .base on the spot and discovers that an earthquake was the reason 
for the disappointing sales of the previous car. 
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<RTKL> 
<HEM>> 
<TITL£> 

Stdrf i re : Boardroom Comicbook 

</TlTLE> 

</HEAD> 

<BODY> 

<8l> 

Boardroom Meeting 
</Hl> 

<H3>Click here for the <A QR£r-soundtr<jck8 .du><EH>30undt raclt . </EM></A></0 

3> 

<P> 

<IMC SROmike.gif ALT-'Photo of Mike claiming that Julie's car will not 
sel 1 . "> 

<IMC SRC-jul ie .gi f HXIGHT-1"79 WIDTH-24S ALT- ' Phot o . o f Julie searching 

for info on her laptop. "> 
<IHC SRC-laptop .gi f HE1CHT-1"?9 WIDTH-245 ALT- "Photo of Julie' 3 laptop. •> 
<1MC SRC-bigscreen . gif R£ICBT-n9 WIDTH-245 ALT-*Photo of Julie and 

Mike in front of th*e multimedia screen. "> 
<IMG SRC-earthcruake.gif HEICHT-P9 WIDTH-245 ALT-*Photo of ne*spape 
r with earthquake Ailicle . "> 
<H3>The Plot</B3> 

Julie is giving a multimedia presentation to the Board of Directors (where 
some of the members are attending remotely) . She is preposing that the comp 
any start manufacturing a car designed by her division. Her rival, Mike, sp 
rings a surprise during the meeting and claims that Julie's car will not se 
11 ue 1 1 since a previous attempt at a similar car failed. Julie uses her la 
ptop computer to search a remote information base on the spot and discovers 
that ar> earthquake was the reason for the disappointing sales of the prtvi 
ous car. 
<RB> 

Last updated February 22, 1995. 

</BODT> 

</B7ML> 



FIG. 10 



19 



EP0 737 930 A1 



(File v) (View v) (Edit v) C H e'P -0 



00 



File Edil View Go Bookmarks OpUons Directory 



Help J 



B-Kk 




ft 

Hot* 



R«*<w<J 



Open 



ft 



o 

Sroo 



Location: Ihnp: / ^rujgifV57Ejakc>c7slartllre' 



[Questions 



rami 



}NetO<fctofY{ 




>sun 



Technology and Research 



Starfire: A User Interface Prototype 



940 
942 

y 




945 



J 



Slarflrc l$ a film showing how your office may look and work In »hf y«ar 2004 using advanced us«r 
Interface technologies. 
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• The in forma i ion space on (he desk-sized computer screen {shown ai the (op of (his page) 

• Telcrobotic manipulation of remote camera with the "SunPad" 

• Texture mapping of information from a 2-D video onto a 3-D mannequin 
^ A video-telephone call from the family is integrated wiih ihc workspace 



Boardroom scene with telepresence and wireless cyberspace searching 
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Technology Featured in the Scene 

• Meeting room with telepresence for remote members 

• Large screen "for multimedia presentations 

• Laptop computer with chordrd input 

• Wireless connectivity between laptop, library server, and the big screen 

• Bidirectional hypertext links between database items 

^956 
Film Clips ^ 

One minute from the scene is available in these formats: 

-958 
•960 



• QuickTime movie (24 MB)- 

• MPEG movie (3.8 MB)—" ~~ 3UW ^962 

• A "comic book" with still photos and the sound track (0.8 MB) 
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